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(57) ABSTRACT 

In the face of unacceptable data loss on connections between 
a content source and a content consumer, additional con- 
nections therebetween are opened. These additional connec- 
tions are preferably opened between the content source and 
a proxy disposed between the content source and the content 
consumer. The proxy may then seam together data streams 
received from the content source across the additional 
connections before passing a resultant seamed stream to the 
content consumer. The seamed stream may be constructed 
by filling in information gaps in any of the data streams 
received from the content source with content derived from 
others of the data streams received from the content source. 
This derivation may be made on the basis of identifying 
characteristics (e.g., packet sequence numbers and/or time 
stamps) common throughout each of the data streams 
received from the content source. 
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METHOD AND APPARATUS FOR SEAMING a top-level server. In other words, all queries for addresses 

OF STREAMING CONTENT are routed to certain computers, the so-called top-level 

fipt n op thp iisrvFTMnoM servers. The top-level server matches the domain name to an 

ricLU ur i Mt iin vtiN i iuin Ip address of a domain name ^tx capable of directing the 

The present invention relates to a scheme for merging 5 inquiry to the computer hosting the sought after Web page 

together information from multiple input data streams to (or other content) by matching an alphanumeric name such 

produce an output data stream that includes fewer informa- as www.domain.com with its numeric IP address, 

tion "gaps" than any of the individual input data streams. In In addition t0 Web md ^ ^ more and more 

one example, the scheme is applied to live broadcasts of Imernet users are accessing mu i time dia content (e.g., files 

streaming content delivered via the Internet. ao mat mclude ^ quality & ^ cai imageSj movies md/oT 

BACKGROUND sound). This creates difficulties because such files are usu- 

The Internet is a vast and expanding network of networks J Uv <l uite ^jj? the bandwidth available through the 

of computers and other devices linked together by various lnU ^ 1 1S llimted -. ^f* in or6 * T l ° make m^cdia files 

telecommunications media, enabling all these computers usable > streamin g 15 often employed, 

and other devices to exchange and share data. Sites on the w i m conventional files (e.g., data files), clients (e.g., Web 

Internet provide information about a myriad of corporations browsers) completely download the requested content 

and products, as well as educational, research and entertain- before viewing it. This technique works well for relatively 

ment information and services. An estimated 30 million smalJ file s, but often suffers from unacceptable (from the 

people worldwide use the Internet today, with 100 million 2Q point ofviewofme user) delays when large multimedia files 

predicted to be on the "net" in a matter of years. are involved. Streaming is the term given to a technique 

A computer or resource that is attached to the Internet is wherein a client downloads a portion of a file, decompresses 

often referred to as a "host." Examples of such resources ( if necessary) that portion, and starts playing the contents 

include conventional computer systems that are made up of thereof (e.g., audio and/or video) before the rest of the file 

one or more processors, associated memory (typically vola- 25 arrives. A buffer of information is built up before playback 

tile and non-volatile) and other storage devices and periph- starts > s° ^ to prevent underflows if the remaining data is 

erals that allow for connection to the Internet or other delayed during transmission. Furthermore, subsequent por- 

networks (e.g., modems, network interfaces and the like). In tions of lhe multimedia file are downloaded during playback 

most cases, the hosting resource may be embodied as to keep me buffer relatively mU. Triis technique thus accom- 

hardware and/or software components of a server or other modates the downloading and playing of large multimedia 

computer system that includes an interface, which allows for files ™&o\it incurring lengthy delays before the content is 

some dialog with users thereof. Generally, such a server will available for viewing. 

be accessed through the Internet (e.g., via Web browsers Multimedia files are often transported over the Internet 
such as Netscape's Navigator™ and Communicator™ and using special transport protocols. For example, the real-time 
Microsoft's Internet Explorer™) in the conventional fash- 35 transport protocol (RTP) provides delivery service for mill- 
ion . timedia applications and also provides means for multimedia 
Briefly, if an Internet user desires to establish a connection applications to work over networks. RTP does not, however, 
with a host (e.g., to view a Web page located thereat), the provide guaranteed or in-sequence delivery (and hence it is 
user might enter into a Web browser program the URL (or referred to as an unreliable transport protocol), but does 
Web address) corresponding to that host. One example of 40 provide a packet sequence number that can be used to detect 
such a URL is "http://www.domain.com". In this example, missing packets and to reconstruct an original transmission 
the first element of the URL is a transfer protocol (most sequence. 

commonly, "http" standing for hypertext transfer protocol, RTP usually carries data in the form of packets, using the 

but others include "mailto" for electronic mail, "ftp" for file user datagram protocol (UDP) as the delivery mechanism, 

transfer protocol, and "nntp" for network news transfer 4S UDP provides a "wrapper" around data packets, with the 

protocol). The remaining elements of this URL (in this case, wrapper providing for multiplexing and demultiplexing as 

"www" standing for World Wide Web — the Internet's well as error checking services. Essentially, a UDP packet is 

graphical user interface — and "domain.com") are an alias made up of a UDP header and UDP data encapsulated as the 

for the "fully qualified domain name" of the host. data portion of an IP packet. The IP packet itself includes an 

Each fully qualified domain name, in its most generic 50 Ip header (which includes the address information discussed 

form, includes three elements. Taking "computer.host.com" above) as well as the user data (i.e. the multimedia content 

as an example, the three elements are the hostname of interest) as a payload. 

("computer"), a domain name ("host") and a top-level In some cases, RTP is used with other protocols, such as 

domain ("com"). Further, each fully qualified domain name the transmission control protocol (TCP). Unlike UDP, TCP 

is unique throughout the Internet and corresponds to a 55 provides a reliable, error-free, fiillKhiplex channel between 

numerical Internet protocol (IP) address. IP addresses facili- two computers. TCP uses IP to transfer data, but provides 

tate communications between hosts and clients in the same mechanisms to take care of lost or duplicated IP datagrams 

way that physical addresses (e.g., 123 Main Street, (i.e., packets) and to ensure proper sequencing thereof. Thus, 

Anytown, Anycity) facilitate correspondence by mail. Each TCP provides reliable end-to-end transport, ensuring that 

IP address is made up of four groups of numbers separated eo what is received is an exact duplicate of what is transmitted, 

by decimals. Thus, in the case of the hypothetical host Where broadcasts of "live" information are concerned, 

"computer.domain.com", the corresponding IP address RTP usually uses UDP as the delivery transport. This is 

might be 123.456.78.91. A given host looks up the IP because TCP may introduce unacceptable delays in receipt, 

addresses of other hosts on the Internet through a system Further, when an application starts an RTP session, a second 

known as domain name service. 65 p ort for communication according to the real time control 

Thus, once a URL is entered into a browser, the corre- protocol (RTCP) is opened. RTCP works in conjunction with 

sponding IP address is looked up in a process facilitated by RTP to provide flow control and congestion control services. 
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The idea is that the exchange of RTCP packets between a 
client and server can be used to adjust the rate of transmis- 
sion of the RTP packets, etc. 

Associated with RTP is the real time streaming protocol 
(RTSP). RTSP is a client-server multimedia presentation 
control protocol that control functionality such as content 
type, interactive stream control, error mitigation, bandwidth 
negotiation, multicast, live broadcasting and monitoring. 
Just as HTTP transports HTML (hypertext markup 
language — an instruction set that allows an HTTP client to 
render a desired image, etc.), RTSP handles data. The 
difference is that while HTTP clients always make requests 
and HTTP servers always service those requests, RTSP is 
bi-directional, with both servers and clients making requests 
and servicing them. RTSP accomplishes data transfer using 
TCP or UDP. 

Thus, RTSP is a generally a TCP connection over which 
commands are sent and responses are returned. Clients 
negotiate data channels with servers via SETUP commands. 
These channels typically specify that data will be sent as 
RTP/RTCP "packets", but the transport type may be speci- 
fied and/or modified as part of the negotiation. Further, the 
negotiations may determine whether the packets will be sent 
over TCP (e.g., as binary packet data imbedded in an RTSP 
command stream via an escape sequence) or UDP (e.g., as 
true packets). 

The RTP portion of a channel contains actual media data 
for single stream flows (e.g., compressed audio data). In 
contrast, an RTCP portion of a channel (which typically is 
assigned one UDP port number or TCP channel number 
larger than the RTP port number or channel — for example, 
UDP port 6970 for RTP and 6971 for RTCP) usually 
contains clock-synchronization data and client-server 
control/status messages. As indicated above, RTP data typi- 
cally flows in one direction, from the server to the client. 
RTCP packets are typically sent in both directions, for 
example as client status report messages and server status 
report messages. 

The following dialog illustrates (from a client's point of 
view) a portion of an RTSP session wherein a media 
description is retrieved and the single audio stream specified 
thereby is played. Those of ordinary skill in the art will 
recognize that this is a "live" stream as opposed to prere- 
corded content, as is evident from the absence of a "range" 
tag in the session description protocol (SDP). In the initial 
client-to-server communication, a request to describe the 
stream located at a particular Web address (rtsp:// 
qt.macroradio.net/gogaga) is sent, along with suggested 
connection parameters, such as a bandwidth: 
Send data (130 bytes). 

<00000000<DESCRIBE rtsp://qt.macroradio.net/gogaga 
RTSP/1.0 

<00000033<CSeq: 1 
<0000003C<Accept: application/sdp 
<00000055<Bandwidth: 112000 
<00000068<User-Agent: QTS/1.0b22 
<00000080< 

The server responds with a description of the stream as part 
of an SDP: 

Receive data (566 bytes). 
>00000000>RTSP/1.0 200 OK 
>00000011>Server: QTSS/v65 
>00000023>Cseq: 1 

>0000002C>Content-Type: application/sdp 
>0000004B>Content-Base: rtsp://qt.macroradio.net/ 65 
gogaga/ 

>0000007B>Content-length: 420 



25 



30 



35 



40 



45 



50 



55 



60 



>00000090> 
>00000092>v-0 

>00000097>oo3134316689 3134906918 IN IP4 
192.231.139.83 

>000000C7>s=GoGaGa Brand Radio 
>OOOOO0DD>i=«A9»1999— All rights reserved - 
>000000FF>u=http://www,macroradio.net 
>0000011C>a=x-qt-text-nam:GoGaGa Brand Radio 
>00000140>a=x-qt-text-cpy:«A9»1999 — All rights 
reserved 

>0000016D>a=x-qt-text-cpy:(c) 1999 ERC: The Eclectic 

Radio Company, 

>000001A7>LLC. 

>000001AD>t-3134316689 3134320289 
>000001C6>c-IN IP4 0.0.0.0 
>000001D8>a«control: * 
>000001E5>m=audio 0 RTP/AVP 97 
>000001FB>a=rtpmap:97 X-QT 
>0000020D>a«x-bufferdelay: 10 
>00000221>a=control:trackID=l 

The client then defines conditions for a playback, using a 

specified port: 

Send data (143 bytes). 

<00000082<SETUP rtsp://qt.macroradio.net/gogaga/ 
tracklD=l RTSP/1.0 
<OOOOO0BC<CSeq: 2 

<000000C5<Transport: RTP/AVP;unicast;client port- 
6970-6971 

<OOO0O0F7<User-Agent: QTS/1.0b22 
<0000010F< 

The server responds with the port address from which it will 
play: 

Receive data (165 bytes). 
>00000236>RTSP/1.0 200 OK 
>00000247>Server: QTSS/v65 
>00000259>Cseq: 2 

>00000262>Session: 1411920655;timeout«60 

> 0000028 2 >Transport: rtp/avp; source- 

192.231.139.182;server_port-2000-2001; 

>000002C2>client port-6970-6971 

>000002D9> 

The client then initiates play back using the specified port 
addresses: 

Send data (125 bytes). 

<00000111<PLAY rtsp://qt.macroradio.net/gogaga RTSP/ 
1.0 

<00000140<CSeq:3 
<00000149<Range: npt=0.000000— 
<0000015F<Session: 1411920655 
<00000174<User-Agent: QTS/1.0b22 
<0000018C< 

In response, the server begins playing the selected stream: 

Receive data (92 bytes). 

>000002DB>RTSP/1.0 200 OK 

>000002EC>Server: QTSS/v65 

>000002FE>Cseq: 3 

>00000307>Session: 1411920655 

>0000031C>RTP-Info: url«trackID=l 

>00000335> 

At this point, RTP media data and RTCP control packets 
will start flowing between the specified UDP ports, i.e., RTP 
data from server port 2000 to client port 6970, and RTCP 
packets between server port 2001 and client port 6971. Note 
that the server's address 192.231.139.182 can be seen in the 
SETUP response above. An example of an RTCP packet 
transmitted from the client (port 6971) to the server (port 
2001) is shown below: 
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Send packet data to port 1 (84 bytes). 

<O0000OO0< 80 C9 00 01 00 00 62 76 81 CA 00 12 00 00 62 76 bv. bv 

<00000010< 01 0D 51 54 53 20 38 30 35 32 31 38 36 39 33 02 ..QTS 805218693. 
<OO0O0O20< 13 51 75 69 63 6B 54 69 6D 65 20 53 74 72 65 61 .QuickTime Strea 
<OOOOOO30< 6D 69 6E 67 06 1A 51 75 69 63 6B 54 69 6D 65 20 ming. .QuickTime 
<00000040< 53 74 72 65 61 6D 69 6E 67 2F 31 2E 30 62 32 32 Streaming/1. 0b22 
<00000050< 00 00 00 D2 



If one were to decode the data, it may translate to "client 
received 5% loss and 123456 bytes and client "name" is 
'QuickTime Streaming' client". 

An RTCP packet sent from the server (port 2001) to the 
client (port 6971) may resemble the following: 



between the content source (e.g., the server) and the content 
consumer (i.e., the client), depending on the network traffic 
conditions. If this loss is high (e.g., 10% or more), the 
viewing quality can be degraded to the point where it is 
unacceptable to a user. 



Receive packet data from 19X231.139.182:2001 (108 bytes). 

>00000000> 80 C8 00 06 00 00 45 38 BA F0 68 00 42 IF 8E 44 E8..h.B..D 

>00000010> 3D 25 45 EB 00 E7 A3 3F BB 8C 3A 78 81 CA 00 13 =%E....?..:x.... 

>00000020> 00 00 45 38 01 18 51 54 53 40 6C 6F 63 61 6C 68 ..E8..QTS@localh 

>00000030> 6F 73 74 20 31 35 36 31 36 31 35 34 36 39 02 13 est 1561615469.. 

>00000040> 5 1 75 69 63 6B 54 69 6D 65 20 53 74 72 65 61 6D QuickTime Stream 

>00000050> 69 6E 67 06 13 51 75 69 63 6B 54 69 6D 65 20 53 ing. .QuickTime S 

>00000060> 74 72 65 61 6D 69 6E 67 00 FF 00 40 treaming...@ 



Once decoded, this information may state, "RTP time t 
means universal (wall clock) time y and server "name" is 
"QuickTime Streaming server". This is essentially a refer- 
ence to an absolute reference time source used by the server. 

The actual RTP media packets transmitted from the server 
(port 2000) to the client (port 6970) may resemble the 
following: 



Others have attempted to solve this problem (improved 
viewing quality or, more .generally, improved user experi- 
ence in the face of data loss), but have primarily concen- 
trated on trying to make the transport of information from 
source to client more reliable. Thus, for example, others 
have attempted to use TCP rather than UDP as the trans- 
mission protocol. Although TCP guarantees that all packets 



Receive packet data from 191231.139.182:2000 (200 bytes). 
>00000000> 80 El BC 73 3D 21 F8 IB 00 00 45 38 14 00 80 01 ...s-!....E8.... 
>00000010> 02 B6 4B 19 09 14 80 9E 5D 26 35 24 88 64 2A 20 ..K...]&5$.d* 
>00000020> DO C2 98 16 92 55 41 9C 82 46 16 35 9D A8 D7 27 ....UA..R5...' 

>00000030> 13 1A B7 37 D6 E4 05 5B 40 AF E7 11 D3 84 9C B8 ...7...[@ 

>0000GO40> 45 8D 51 01 Fl A4 C5 97 0B 58 88 2A 4A Dl C4 13 E.Q X.-J... 

>00000050> FC 8C 58 A5 46 8A A2 3B 63 66 6F 23 2F 38 IB 61 ..X.F..;cfo#/8.a 

>00000060> 0B 15 2A D3 49 22 C9 98 C8 OF 16 40 1A 53 9D A8 ..M M @.S.. 

>00000070> 79 Fl CE EE C6 19 Bl 26 C5 A8 CB 4D 4B 3B F3 73 y. &...MK;.s 

>00000080> 4C 6A 33 5F D3 5F 2C 46 60 84 CO 08 14 14 26 EC Lj3_._F &. 

>00000090> 5E DF 49 49 48 B5 B3 02 5F 88 F5 EC 29 10 AB 72 .IIH..._..)..r 
>OO0O0OA0> A6 D8 3E D4 9A D2 14 2A 6F 86 AD 22 9E 0B 4C 50 ..>.... *o..\.LP 
>OO0O0OB0> 5C BC 0B 88 6D 13 0C 34 3C 44 CB 92 BB 6B IB 18\...m..4<D...k.. 
>0O000OC0> 51 1C 7D 12 01 00 00 00 Q.} 



Finally, upon conclusion of the playback or at some other 
point, the client may decide to quit, so an instruction is 
passed to server over RTSP (the TCP connection is still open 
during the playback) to stop everything on this "session": 55 
Send data (107 bytes). 

<0000018E<TEARDOWN rtsp://qt.macroradio.net/gogaga 
RTSP/1.0 

<000001Cl<CSeq: 4 

<000001CA<Session: 1411920655 60 

<000001DF<User-Agent: QTS/1.0b22 

<000001F7< 

Although these and other transport protocols exist, there 
still exist problems with the viewing of streaming content 
over public networks or networks of networks such as the 65 
Internet. For example, whenever unreliable transport proto- 
cols (e.g., RTP) are used, there can be significant data loss 



that make up a file will arrive and will be in sequence when 
ultimately played out, it does so by requesting retransmis- 
sions of any lost packets. This not only introduces delay into 
any playback (e.g., while waiting for lost packets to be 
retransmitted), it also adds to the total volume of network 
traffic thus leading to still further packet loss due to con- 
gestion and the like. The delays introduced by the use of 
TCP often mean that the ultimate user experience is poor, 
and perhaps even less acceptable than would be the case if 
lost packets were simply overlooked. 

Other solutions have proposed using network bandwidth 
reservation protocols in place of unreliable transmission 
protocols. Unfortunately, such protocols are not always 
available end-to-end, and so this solution is not always an 
option. 

Another strategy for dealing with the loss of data between 
the source and the requesting client is to control the amount 
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of data being transmitted. Where possible, clients that are FIG. 1 illustrates a situation in which individual users 
experiencing significant data loss may instruct the server to have connected to a content source using independent con- 
send less data. Thus, in the scenario of a streaming movie, nections across the Internet; 

the server may be instructed to send only key frames and not FIG. 2 illustrates the use of proxies disposed in commu- 
any (or perhaps just a few) difference frames. However, this 5 nication paths between a content source and one or more 

strategy is only successful where the data loss is due to content consumers in accordance with an embodiment of the 

actual bandwidth overloading at the client and not due to present scheme; 

other factors in the intervening network. For example, if the FIG. 3 illustrates a seaming operation from a conceptual 

packet loss is due to consistent buffer overflow at the standpoint in accordance with an embodiment of the present 
receiver, instructing the server to send less data may prevent 10 scheme* 

such overflows and provide a better user experience. Where, uj^ a n ♦ # r>™ , , . , , 
. 1*1* j ii *_ i FIG. 4 illustrates an RTF packet header showing a 

however, packet loss is due to overall network congestion, . £ . , , . . A . b 

. t t . r ,, 4 j, j .„ . %r A sequence number field useful in seaming operations per- 

lnstructing the server to send less data will have no effect on r „ j * j •**_ j- . c *i_ . 

iL . ° . , . . . ... j . . lormed m accordance with an embodiment of the present 

the packet loss, because the insignificant reduction in the scheme* 

total number of packets within the network due to the stream 15 ' „ 

under consideration will not be sufficient to dramatically c mG ' 5 ^strates a functional depiction of a proxy con- 
affect the packet loss rate for that stream. Tne net result will fiff™* to pe*nn seammg operations m accordance with an 
simply be roughly the same packet loss rate over a fewer embodiment of the present scheme; and 
number of transmitted packets — leading to an even worse & illustrates a sequence of operations to be per- 
condition. 20 f° rme d by a sequencer for the proxy shown in FIG. 5 in 
What is needed, therefore, is a new solution to the accordance with an embodiment of the present scheme. 

problem of dealing with data loss during the broadcast of A ¥I ™ nronnnmnKr 

f- * ■ * * DETAILED DESCRIPTION 

live, streaming content. 

Disclosed herein is a scheme for seaming together mul- 
SUMMARY OF THE INVENTION 2 5 tiple streams of streaming content broadcast over a public 

In one embodiment, two or more data streams, each made network or netwo * of networks such 1 as * he Intern f In 

up of a number of packets, that are received from a content ^ ssence ' a P rox y (**»P™nt or explicit) is introduced 

source (e.g., a server) across one or more computer networks b J? WBC ° a extern source (e.g., a server) and one or more 

(e.g., the Internet) using an unreliable media transmission che ? te < c *' Web browsers > or plug-ins therefor, configured 

protocol (e.g., RTP) are seamed together at a proxy disposed 30 to P la * str f amm f or otber ^tim^dia viewers, other 

between the content source and one or more content con- Pixies playback devices, etc.), preferably at a location that 

sumers. This allows for the provision of one or more output » close ^ V h y^y or logically) to the clients. Herein, 

data streams to the one or more content consumers (e.g., web ^ e lerm f T V * ™™ { l ° deSCrib f. and/or / efer to a dev ? ce 

browsers) that include fewer missing packets than any mat resides logically between a client and server, or other 

individual one of the data streams being received at the 35 content source, and mat processes mformation flov^g there 

proxy from the content source. The seaming operation between in some manner. Proxies may be physically 

generally includes packets from at least one of the data <f loc f d with clients and/or servers and/or may be stand- 



streams received from the content source in the output data 



alone devices. Data streams from the source are received at 

streams * ~ - - — " iV the proxy and from there are routed to the requesting clients. 

_ ' , , , . ,,..1 40 En route, each stream can be buffered and any information 

In a further embodnnent, one or more additional connec- ( due t0 ^ loss) m me received streams CM be 

Uons between a content consumer and a content source are filled ^ information from other server-source streams. A 

opened m response to an i indication that data loss between resuhm .^^^ stream can be provided from tne pro 

the content source and the content consumer over one or t0 one of more of , he clients> ^ the sbeam hayi 

more exiting connections therebetween has reached an 4J fcwe , Morm!ltion man of ^ received streams 

unacceptable level (e.g., as defined by a threshold). Tlese from the M additional benefit of ^ h is 

additional connections are preferably opened between the that mu]ti le clients share me ^ ^ 

content source and a proxy disposedbetween the content redudn , he overall (raffic flow ^ ^ omerwise b(J 

source and the content consumer Hie proxy may seam experienced if me multiple c]ients were each to open a 

together data streams received from the content source separate session with the content source. Even where only a 

across the additional connections before passing a resultant sin ^ e cliem ^ requesting , he streaming content, the present 

seamed stream to the content consumer. The seamed stream scheme ides a new a h in da(a deU b 

may be constructed by filling ,n information gaps in any of fa for multi k concurrent (each h ^ a 

the data streams received from the content source with differem romlmmication path) t0 ^ opened ^ me 

content denved from others of the data streams received JS source> aUowin fof m6 a h t£) be ^ 

from the content source. This derivation may be made on the AU . ... , c A ^ • •« * ^ j 

u . f »j • u / i* Although discussed with reference to certain illustrated 

basis 01 ldentitymg cnaractenstics (e.g., packet sequence j> * * n t r 

, j/ *• * \ >L t_ . u c embodiments, upon review of this specification, those of 

numbers and/or time stamps) common throughout each of . / tL _ .„ . , A ' 

iU „ . _ 1 r_ t ^ & ordinary skill in the art will recognize that the present 

the data streams received from the content source. « J n . ■ • /* , 

scheme may find application in a variety of systems, perhaps 

Other features and advantages of the present invention 60 ^ one or more ndooT variations . Therefore, in the fol- 

will be apparent from the following discussion. lowing description the Ulustrated embodiments should be 

BRIEF DESCRIPTION OF THE DRAWINGS ? garded as ^P 1 ^ ^ nly ^ shoi » ld DOt be deemed to be 

limiting in scope. Further, it should be kept in mind that 

The present invention is illustrated by way of example, some portions of the detailed description that follows are 

and not limitation, in the figures of the accompanying 65 presented in terms of algorithms and symbolic representa- 

drawings in which like reference numerals refer to similar tions (e.g., through the use of flow diagrams, etc.) of 

elements and in which: operations on data within a computer memory. These algo- 
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rithmic descriptions and representations are the means used Unfortunately, simply introducing the proxy 22 into the 
by those skilled in the computer science arts to most path between multiple users and the content source may not 
effectively convey the substance of their work to others be sufficient to bring packet loss to a point where optimum 
skilled in the art. viewing conditions are available for the users. Depending 
An algorithm is here, and generally, conceived to be a s upon the overall traffic conditions within network 20, there 
self-consistent sequence of steps leading to a desired result. may still be significant packet loss on connection 28 
The steps are those requiring physical manipulations of between server 12 and proxy 22. Thus, a further aspect of the 
physical quantities. Usually, though not necessarily, these present scheme is the introduction of data seaming, 
quantities take the form of electrical or magnetic signals Data seaming is a counter-intuitive process by which, in 
capable of being stored, transferred, combined, compared 10 the face of significant data loss, even more data than was 
and otherwise manipulated. It has proven convenient at originally being broadcast is requested. This is counter- 
times, principally for reasons of common usage, to refer to mtu itive because the conventional approach to dealing with 
these signals as bits, values, elements symbols, characters, data loss is t0 reduce me amount data ^ transmitted 
terms, numbers or the like. It should be borne in mind, between ^ dient in the h that {h]& ^ reduce 
however tha aU of these and similar terms are to be tioil md allow for ^ d communication . Data 
associated with the appropriate physical quantities and are 13 . . , r i_ j *. « 

i • • i £ i v j * *l seaming takes the opposite approach and actually causes 

merely convenient labels applied to these quantities. . & - . K* , , , , , ' 

w , c ii , , i . . . j .... more mformation to be downloaded from the source, with 

Moreover, unless specifically stated otherwise, it will be tU i u • * *■* u * *i_ i * * 

appreciated that throughout the description of the present * h * g0a !* m & ° Shtch l 0 ^"' ° r . seam « P ackets from 

scheme, use of terms such as "processing", "computing, d f™ Dt f ut s,rea , mS 0r . fraJSc floW ? mt ° an OU ^ Stream 

"calculating", "determining", "displaying", "rendering" or » that has &wer information gaps man any of the input 

the like, refer to the action and processes of a computer streams. 

system, or similar electronic computing device, that manipu- FIG. 2 illustrates this concept. In the face of significant 

lates and transforms data represented as physical (electronic) packet loss over connection 28 (e.g., when the number of 

quantities within the computer system's registers and memo- missing packets per interval of unit time reaches a 

ries into other data similarly represented as physical quan- 25 threshold — a quantity which can be determined using the 

tities within the computer system memories or registers or sequence numbers present in the packet headers, etc.), proxy 

other such information storage, transmission or display 28 opens one or more additional connections 30a and/or 306 

devices. Again, these are the terms and descriptions com- to server 12. Each of these new connections 30a and/or 306 

monly used by and among practitioners of ordinary skill in (of which there may be one or more depending upon traffic 

the relevant arts. 30 conditions, the number of clients being served, the number 

Turning now to FIG. 1, some further details regarding the of packets being lost on the primary connection 28, etc.) will 

basic problem outlined above can be stated. In general, when transport the same data that is being transported across 

seeking to view live, streaming content over an Internet or connection 28. However, because the packet loss on each 

other network connection, a user 10 will connect to the connection 28, 30a and 306 is a random or pseudo-random 

content's source (e.g., server 12). This connection 14 will 35 process, it is likely that packets that are dropped over 

allow for the transport of the streaming content, usually connection 28 will appear over at least one of connections 

according to one of the protocols for multimedia transmis- 30a and 306. Thus, the information gaps caused by the 

sion discussed above. Now, when a second user 16 wishes missing packets from connection 28 can be filled in using the 

to view the same broadcast, he or she will open a separate packets from one of these new connections 30a and/or 306. 

connection 18 across the network (e.g., the Internet) 20 to 40 Then, the resulting output streams that are played out to the 

server 12. Thus, the content that is being downloaded by users 10 and 16 over connections 24 and 26 can be the 

user 10 is the same content that is being downloaded by user "seamed" streams that include packets from multiple ones of 

16. This duplication of material adds to network congestion the proxy-server connections 28, 30a and 306. 

and (especially as this scenario is repeated many times over FIG. 3 further illustrates this seaming process. Assume 

for further users) can contribute to packet loss on each of the 45 that two information streams 40 and 42 are inbound to a 

connections. proxy from a content source. These streams 40 and 42 may 

To combat this and the other problems described above, be (and in some cases preferably are) routed over different 

the present scheme introduces a proxy between the content paths within network 20. Thus, it can be expected that the 

source (e.g., server 12 or another proxy) and the users 10 and packet losses on each of the two streams will be different. 

16, as shown in FIG. 2. Proxy 22 is introduced in the 50 Importantly, the two streams 40 and 42 represent the same 

connection between content source and content users, pref- content and the individual packets 44 that make up the 

erably as close to the last physical media link to the users as streams are uniquely identified in some fashion. For 

possible. Thus, the proxy 22 may be situated at the point a example, where the packets 44 are transported according to 

user's dial up Internet connection is terminated (e.g., as RTP (as is the case for much streaming content on the 

deployed by an Internet service provider as part of a modem 55 Internet), u nique sequence numbe rs (shown as letter desig- 

bank). Now, when a user (or more particularly a client nators A, B;C, ... in the figure) are available to differentiate 

application being run on a computer platform operated by a the packets from one another. Because the streaming content 

user) connects to server 12, the connection actually passes is "live", the same sequence numbers or time stamps may 

through proxy 22. Thus, user 10 has a connection (e.g., a dial (typically will) identify the same information packets 44 in 

up connection) 24 to proxy 22, while user 16 has a separate 60 each stream 40 and 42. Note this may not be the typical case 

connection 26 thereto. Streams that are downloaded from fo r the playback of stored cont ent, where diEerent streams 

server 12 may be routed over a single connection 28 to proxy may include data (e.g., starteo^fjom different points within 

22 before being passed to each user 10 and 16 over the a movie, etc) identified by d ifferent sequence number s or 

individual user connections 24 and 26. By reducing the time stamps. Also, even for five content in some cases the 

volume of data being downloaded from server 12 in this 65 sequence numbers may be shifted so time stamps may need 

fashion, the overall network congestion is reduced and fewer to be compared/used to identify corresponding packets 

packet losses may be experienced. within different streams. 
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If the sequence numbers or time stamps of packets of sixteen-bit field that is sequentially incremented for each 

different streams are not the same, they can be normalized RTP packet transmitted by the content source. Other trans- 

before the seaming operation is performed. For example, an mission protocols may include similar identifying sequence 

RTCP stream associated with downloaded content may numbers or representations that can be exploited to allow for 

include timestamp information that is transmitted periodi- 5 seaming together two or more information streams, 

cally. This timestamp information is correlated to an abso- T , . , A . , , 

lute reference clock (e.g., the wall clock noted above) for a In 0lher ca f S ' * P"**? 1 , hc * dcT . m ^ not mcl / lde a 

particularserver,soonecande^ sequence number, but may include a timestamp, such as is 

for a particular stream and/or a number of streams. By found in timestamp field 54. For RTP packets, the timestamp 

comparing these different offsets one can then align similar in 15 a ^y-two-bit value that reflects the sampling interval of 

packets and retrieve sequence number offsets therefor for me first octet m the RTP data P acket * ^ value * derived 

each stream. In this way, the streams can be aligned. Of from a clock source at the content source. Similar timestamp 

course, over time errors (e.g., due to numerical rounding) fields arc used Wlth other transmission protocols and at least 

may be introduced so it may be necessary to compare, 1D lhe ^ of llve streaming content, the timestamp values 

periodically, actual packet contents on a byte-by-byte or . of ^o packets havmg similar data but being transmitted on 

hash value basis or by using some other comparison mecha- different streams and/or connections (logical and/or 

nism or method to maintain this inter-stream synchroniza- Physical) may be equal (or similar enough) so as to permit 

uon the identification of information gaps and corresponding 

By monitoring the sequence numbers in a stream, the ava f abl f P a .* k f ^ msertion Uaerein^ Of course, other 

prox y ca n detectt isiinl ^ckete (shown in dotted outhki n 20 f/T^ , i 1?**^^*™^***^ 

th e figur e) and subs titute a packet from aii o ffier stre am nTthe 2 ° that wiU permit such idenUfication and mserUon, mu^ 

-out put. So, the proxy may begi Spliyi ng out a acam SditeS mg f ° r * e ^ ammg of mfo ™ atloa * a manner 

46 using packets A, B and C from stream 40. Then, upon COQS * tent with the present scheme, 

detecting a missing packet in stream 40, the proxy may look Now turning to FIG. 5, one possible implementation of a 

to stream 42 to provide packet D to the seamed stream. After 25 P rox y 50 ^ illustrated. It should be appreciated that this 

this point, the proxy has the option of returning to stream 40 illustration does not show all of the components that may be 

to look for the next packet (E) in line, or it may continue to needed to allow operation of the proxy in a real network, 

choose packets from stream 42 (as shown in the figure) until Rather, the focus is on the functional components that may 

a gap in that stream is encountered. Regardless of which be used t0 accomplish a data seaming operation, 

option is used, the resulting output seamed stream 46 30 Shown in the figure is a case where multiple input streams 

includes fewer information gaps that the input streams from 52 (e.g., from multiple physical or logical connections to a 

which it is created. content source) are applied to a receive buffer 54. Receive 

In the event none of the input streams have a packet to fill buffer 54 may, in practice, be a shared memory operated 

a gap, the seamed stream will likewise include an informa- under the control of a memory controller that processes the 

tion gap at that point. Nevertheless, if enough input streams 35 incoming streams 52 so as to store data packets thereof in 

are used, and/or those streams are received over sufficiently one or rnore logical queues (which may themselves be 

different routes, it is expected mat me number of information implemented as linked lists of buffers). Thus, the data 

gaps in the seamed stream will remain less than the number packets that make up the input streams 52 are stored in a 

of gaps in any one input stream. As network conditions fashion that allows their respective sequence number (and/or 

improve, the number of alternate input streams (i.e., alter- 40 o tner identifying criteria) and stream/connection to be iden- 

native proxy-source connections) could be reduced. For tified. 

example, one or more lossy streams could be closed down. Sequencer 56, whic h may be a general or spe cial purpos e 

Returning to FIG. 2 then, users 10 and 16 can be provided processor and/or a custom integrated circuit configuredj o 

with seamed streams on connections 24 and 26, respectively. carry out th e sequenc ing operations described hereinr is 

In addition, an additional user 32 that connects to a second 45 responsibleTor examining the various packets from each of 

proxy 34 over a connection (e.g., a dial up connection) 36 the input streams and assembling one or more seamed 

can take advantage of the seamed stream. The proxies 22 and streams within transmit buffer 58. That is, sequencer 56 is 

34 may be communicatively coupled by a connection 38 responsible for collecting and transferring to the transmit 

(e.g., so as to form a virtual private network within network buffer 58 (which again may be a shared memory and/or a 

20), thus providing a communication path from proxy 22 to 50 linked list of buffers), the packets that will make up the 

user 32 that would allow for the reception of the same seamed outgoing stream(s) 60. Packets may be played out of 

seamed stream that is being provided to users 10 and 16. In transmit buffer 58 at a rate optimized for a receiving client 

addition or alternatively, proxy 34 may open yet another under the control of sequencer 56 or a memory controller 

connection 39 to server 12, and provide a further seamed (not shown). In some cases, the seamed stream may also be 

stream (i.e., filling in any gaps in the stream provided by 55 stored in a long-term storage unit 62 (e.g., a memory, tape 

proxy 22 with packets received over connection 39) to user storage unit, CD-ROM or other computer-readable data 

32. Of course, the number of users, proxies and/or connec- storage device). This will facilitate later playback of the 

tions communicatively coupling these elements to one seamed stream (e.g., for later viewing by those who missed 

another and/or to server 12 is variable and is not critical to the "live" broadcast, or even for viewers requesting an 

the broader spirit and concepts involved with the present 60 on-demand instant replay). 

scheme. One example of the manner in which sequencer 56 may 

As mentioned above, where RTP is used to transport be configured is illustrated in the flow diagram of FIG. 6. 

streaming content between the content source and the proxy, Process 70 begins with sequencer 56 resetting internal 

the sequence numbers within the RTP packets may be used counters used for a stream number (step 72) and sequence 

to assist in building a seamed stream. For sake of clarity, 65 number (step 74). These counters will allow the sequencer 

FIG. 4 illustrates an RTP packet header 50, including the 56 to step through the different input streams, looking for a 

sequence number field 52. Sequence number field 52 is a next packet in sequence to apply to the seamed stream in 
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transmit buffer 58. Thus, these counters need only be reset lion of a proxy in a communication path between a content 
for each new seamed stream to be assembled. Note the source and a user. This buffering provides an opportunity for 
counters need not be reset to an absolute starting point (e.g., the proxy to reorder any out-of-order packets before trans- 
0 or 1), rather, they may be reset to the lowest sequence mitting the seamed stream. Thus, the receiving client device 
number value for packets received in any incoming stream. 5 need not cope with out-of-order packets. Also, the transmit 
Once the stream and sequence number counters are rate from the proxy to the user can be adjusted to be optimal 
initialized, the sequence number of the first packet of the for the client device being employed, thus reducing the 
input stream pojmecl to by the stream number counter is possibilities of underflows or overflows at the client. 
examined (step i b) and compared against the sequence Opening additional proxy-server connections can be done 
pumper counTei^alue {step Vh). E ssentially, sequencer 56 is 3Q at any time, regardless of the number of users connected to 
lHJw" be 1 ginning me process oi assembling the output seamed the proxy. In the above discussion with respect to FIG. 2, the 
stream, so the first packet in that seamed stream needs to be example of two concurrent users was used, however, the 
found and passed to transmit buffer 58. Thus, if the value of additional connections 30a and/or 30b could be opened 
the sequence number counter and the sequence number of between proxy 22 and server 12 even if user 10 is the only 
the packet under examination match, then that packet is next u se f requesting the streaming content. Some of these con- 
in line for the seamed stream and it is played out to the nections could even make use of different transport proto- 
transmit buffer (step 80). The process can then continue by cols ( e -S » Tcp ) t0 guarantee packet delivery and/or different 
incrementing the sequence number (step 82) and checking transmission paths (perhaps even through other proxies 
for the next packet in line. locat ^ closer t0 content ^urce) Then, if at a later time 
„„ . - , , , user 16 connects, that user can take advantage of the seamed 
Whenever Ihe value of the sequent number counter and 20 stream for ^ 10 b havj it 6 d licated over 

the sequence number of the packet under examination do not connection 26 

match, this is an indication that a packet is missing from the ^ a scheme for together multi le streams of 

stream under consideration. For example, referring back to streaming broadcast over a public network or net- 

F1G. 3, if the sequencer 56 were examining packets from work of netW orks has been described. Although the forego- 

stream 40 for inclusion in seamed stream 46, the sequence 25 ing description and accompanying figures discuss and iUus-' 

number counter may be initialized to "A". Then, for each of trale spec ifi c embodiments, it should be appreciated that the 

packets A, B and C of stream 40, the sequence number presem inven tion is to be measured only in terms of the 

counter would match the packet sequence number and so claims that follow, 

those packets from stream 40 would be played out to the What is claimed is* 

transmit buffer 58. However, when the sequence number 30 L A met hod, comprising seaming together two or more 

counter was incremented to "D", the next packet examined ^ data streams of common content, each made up of a 

from stream 40 would not have a matching sequence num- number of packe ts, being received from a content source 

ber. Instead, that packet would have a sequence number "E" across one or more compter networks using an unreliable 

(remember the "D" packet is missing from stream 40). media ttansmi5sion protocol at a proxy disposed between the 

Upon detecting this mismatch, the sequencer would incre- 35 content source and one or more content consumers so as to 

ment the stream number counter (step 84, FIG. 6) and provide one or more output data streams to the or more 

examine the next packet from the stream now pointed to content consumers, each of the output data streams includ- 

(step 86). Thus, for the example shown in FIG. 3, the stream mg pac kets from multiple ones of the input data streams and 

counter may have incremented to point at stream 42, and including fewer missing packets than any individual one of 

packet "D" from that stream would have a matching 40 the input data streams being received at the proxy from the 

sequence number. Note, not shown in the flow diagram but content source. 

implied by this discussion is that the sequencer may have to 2. The method of claim 1 wherein the transmission 

examine some or all of the incoming streams in parallel so protocol comprises real-time transmission protocol (RTP). 

as to keep its pointers in the various streams lined up. In any 3. xh e me thod of claim 1 wherein at least one of the 

event, having found the next packet in sequence, packet "D" 45 content consumers comprises a plug-in for a Web browser, 

from stream 42 may be played out to the transmit buffer 58. 4, jfo method of claim 1 wherein the seamed data streams 

If the next stream in line did not have the sought after are recorded as a seamed stream at the proxy, 

packet, the stream number could be further incremented if 5. The method of claim 1 wherein the proxy is physically 

other input streams were available (step 88). Otherwise, if co-located at one of the content consumers, 

no further streams were available, then the sequence number 50 6. The method of claim 1 wherein the content source 

counter would be incremented (step 90) and the seamed comprises a second proxy. 

stream would simply contain a gap in packet sequence 7. A method, comprising opening one or more additional 

numbers. Note that using the process illustrated in FIG. 6, connections between a content consumer and a content 

sequencer 56 chooses packets from a stream until a missing source in response to an indication that data loss between the 

packet is detected, and then switches streams (as shown in 55 content source and the content consumer over one or more 

FIG. 3). An alternative approach would be to recycle back to existing connections therebetween has reached an unaccept- 

the same starting stream each time and include the next able level. 

packet in sequence from the first queried stream that 8. The method of claim 7 wherein the unacceptable level 

includes the desired packet. Still a further approach would is defined by a threshold. 

be to examine each stream (or some number thereof) in 60 9. The method of claim 7 wherein the one or more 

parallel for the next packet in sequence and choose the additional connections are opened between the content 

desired packet from any stream that included it (perhaps on source and a proxy disposed between the content source and 

a round robin or even a priority basis). Any of these the content consumer. 

approaches may be implemented and each is within the 10. The method of claim 9 wherein the proxy seams 

broader spirit and scope of the present seaming scheme. 65 together data streams received from the content source 

An additional advantage provided by the present scheme across the additional connections before passing a resultant 

is the short buffering interval experienced through the inser- seamed stream to the content consumer. 
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11. The method of claim 10 wherein the proxy constructs 21. The proxy of claim 19 being further configured to 
the seamed stream by filling in information gaps in any of seam together data streams received from the content source 
the data streams received from the content source with across the additional connections before passing a resultant 
content derived from others of the data streams received seamed stream to the one or more content consumers, 
from the content source. 5 22. The proxy of claim 21 being further configured to 

12. The method of claim 11 wherein the content is derived construct the seamed stream by filling in information gaps in 

from others of the data streams on the basis of identifying any of me dala streams received from the content source 

characteristics common throughout each of the data streams ^ content derived from mhers of lhe data streams 

received from the content source. ^ , fr ^ m „ t . a 

^ _ . , * , . , e . received from the content source. 

13. The method of claim 12 wherein the identifying 10 r™ f , . , . +u . # . , . . 

. , . J 6 23. The proxy of claim 22 wherein the content is derived 

characteristics comprise packet sequence numbers. £_ iL c , 4 .i_ ^ • r *j *-r * 

a *i_ j r i • i>» u • .1 • i » from others of the data streams on the basis of identifying 

14. The method of claim 12 wherein the identifying t . . , , „ , J & 
characteristics comprise time stamps. characteristics common throughout each of the data streams 

15. The method of claim 12 wherein the identifying received from the conlent source ' 

characteristics of each of the data streams received from the is 24 ^ P roxv of claim 23 wherein the identifying char- 

content source are normalized prior to any comparisons. actenstics comprise packet sequence numbers. 

16. The method of claim 7 further comprising closing 25 ; proxy of claim 23 wherein the identifying char- 
down one or more of the connections between the content actenstics comprise time stamps. 

consumers and the content source. 26 - ^ P rox Y of claim 21 bein S further configured to 

17. The method of claim 7 wherein one or more of the 20 close down one or more of the connections between the 
additional connections comprise alternative network routes content consumers and the content source. 

to the content source. 2 ^- ^ e P roxv °f claim 21 wherein one or more of the 

18. The method of claim 17 wherein at least one of the additional connections comprise alternative network routes 
alternative network routes includes a proxy. t0 the content source. 

19. A proxy configured to be coupled to one or more 25 28 - ™ e P rox y of claim 27 wherein at least one of the 
content consumers and a content source and further config- alternative network routes includes another proxy. 

ured to open additional connections to the content source in 29 ' ^ P roxv of claim 23 farther configured to record the 

response to an indication that data loss between the content seamed data streams. 

source and one or more of the content consumers over one 30 ^ P rox y of claim 23 wherein the identifying char- 

or more existing connections therebetween has reached an 30 acteristics of each of the data streams received from the 

unacceptable level. content source are normalized prior to any comparisons. 

20. The proxy of claim 19 wherein the unacceptable level 

is defined by a threshold. ***** 
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